Image processing apparatus and on-vehicle camera apparatus

ABSTRACT

An image processing apparatus includes a line buffer, an FIR filter serving as an edge enhancement filter that enhances high-frequency components of an image, and an IIR filter serving as a low-pass filter that reduces noise. The same line buffer is used by both the FIR filter and the IIR filter. According to a switching signal, switches enable the FIR filter and disable the IIR filter for a relatively low noise level while the switches enable IIR filter and disable the FIR filter for a relatively high noise level.

TECHNICAL FIELD

The present invention is generally directed to an image processingapparatus that processes an image obtained by using an imaging devicesuch as a charge-coupled device (CCD) and to an on-vehicle cameraapparatus that includes the image processing apparatus.

BACKGROUND ART

Demands for on-vehicle camera apparatuses for use in blind cornermonitors and rear-view monitors on vehicles have increased in recentyears. An image-capturing environment for such an on-vehicle cameraapparatus is not always bright; i.e., the image-capturing environment issometimes relatively dark. Because the brightness of an image framecaptured in a relatively dark image-capturing environment isinsufficient, against which some countermeasure is required.

In a camera apparatuses that includes an imaging device such as a CCD,when the brightness of the image frame is insufficient, it is typical toadopt a scheme of increasing the gain of an automatic gain controlcircuit (AGC) of the imaging device thereby obtaining a brighter image.Such a technology has been disclosed, for example, in Japanese PatentApplication Laid-open No. H4-247777. Meanwhile, increase in the gaindisadvantageously leads to increasing in the noise. As a countermeasurethereagainst, conventionally a scheme of enabling a low-pass filter in adark portion of the image frame where the brightness is not sufficientthereby reducing noise has been generally employed. A low-pass filterhas also been generally implemented by using an infinite impulseresponse (IIR) filter in circuit of relatively small scale. Such atechnology has been disclosed, for example, in Japanese PatentApplication Laid-open No. 2005-184786.

Accordingly, use of an IIR filter on an on-vehicle camera apparatus orthe like leads to the possibility of increasing the gain in the darkportion to maintain the brightness of the image to a desirablebrightness without increasing the noise.

An on-vehicle camera apparatuses or the like typically uses awide-view-angle optical system and includes, in its image processingunit, a high-frequency enhancement filter, such as a finite impulseresponse (FIR) filter, for the purpose of correcting degradation inresolution resulting from the optical system. However, a line bufferwith a capacity of several lines is indispensable for thishigh-frequency enhancement filter. Meanwhile, the scheme of implementinga low-pass filter to suppress noise by using an IIR filter requires anadditional line buffer. However, under recent circumstances wherecapacity necessary for a line buffer has been greatly increased due toincrease in the number of pixels in imaging devices, providing anadditional line buffer for an IIR filter results in increase in circuitscale.

The present invention aims at providing an image processing apparatusthat enhances high-frequency components of an image that are attenuatedby an optical system and reduces noise in a dark portion withoutincreasing circuit scale and that is of low cost and low powerconsumption, and an on-vehicle camera apparatus that includes the imageprocessing apparatus.

DISCLOSURE OF INVENTION

According to an aspect of the present invention, there is provided animage processing apparatus that processes image data obtained by animaging device. The image processing apparatus includes a line bufferthat temporarily and sequentially stores therein the image data; afinite impulse response (FIR) filter that performs shaping of spatialfrequency characteristics of the image data by using the line buffer;and an infinite impulse response (IIR) filter that uses the same linebuffer that is used by the FIR filter as a line buffer for use infeedback of processed image data.

Thus, an additional line buffer is no more necessary for the IIR filter,which leads to reduction in circuit scale.

Specifically, an image processing apparatus that processes an imageobtained by using an imaging device, such as a CCD, and an on-vehiclecamera apparatus that includes the image processing apparatus arecapable of enhancing high-frequency components of the image that areattenuated by an optical system and reducing noise in a dark portionwithout involving increase in circuit scale.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating the overall configuration of animage capturing apparatus according to an embodiment of the presentinvention;

FIG. 2 is a diagram for explaining magnification chromatic aberrationand distortion;

FIG. 3 is a diagram for explaining simultaneous correction ofmagnification chromatic aberration and distortion;

FIGS. 4A and 4B are diagrams explaining independent correction ofmagnification chromatic aberration and distortion;

FIG. 5 is a block diagram illustrating the detailed configuration of amagnification-chromatic-aberration correcting unit illustrated in FIG.1;

FIGS. 6A to 6C are block diagrams illustrating exemplary configurationsof a coordinate transformation unit illustrated in FIG. 5;

FIG. 7 is a block diagram illustrating the detailed configuration of adistortion correcting unit illustrated in FIG. 1;

FIG. 8 is a block diagram illustrating the configuration of a modulationtransfer function (MTF) correcting unit illustrated in FIG. 1;

FIG. 9 is a block diagram illustrating the configuration of a filteringunit illustrated in FIG. 8.

FIG. 10 is an illustration of example coefficients for an FIR filterillustrated in FIG. 9.

FIG. 11 is a block diagram illustrating the configuration of the FIRfilter.

FIG. 12 is a block diagram illustrating the configuration of an IIRfilter illustrated in FIG. 9.

FIG. 13 is a block diagram illustrating how the IIR filter, a linebuffer, and the FIR filter are connected together for a relatively lownoise level.

FIG. 14 is a block diagram illustrating how the IIR filter, the linebuffer, and the FIR filter are connected together for a relatively highnoise level.

BEST MODE(S) FOR CARRYING OUT THE INVENTION

Exemplary embodiments of the present invention are described in detailbelow with reference to the accompanying drawings. One embodiment of thepresent invention provides an image capturing apparatus that captures animage of a subject by using a wide-view-angle optical system with largemagnification chromatic aberration and high distortion occur. The imagecapturing apparatus includes an image processing system that performs,in addition to MTF correction, correction of the magnification chromaticaberration, correction of the distortion, and the like. It is needlessto say that the configuration is not limited to this.

Moreover, in the following explanation, it is assumed that an image iscomposed of additive primary colors, i.e., red (R), green (G), and blue(B). It is needless to say that the present invention can be applied toa case where an image is composed of subtractive primary colors, i.e.,yellow (Y), magenta (M), and cyan (C).

FIG. 1 is a functional block diagram of an image processing system inthe image capturing apparatus according to an embodiment of the presentinvention. The image capturing apparatus further externally includes anoperating unit, an image storage unit, and an image display unit(monitor), which are not shown in FIG. 1. It is assumed that the imagecapturing apparatus is used as an on-vehicle camera apparatus. It isneedless to say that the image capturing apparatus can be used in someother way.

The image capturing apparatus includes a control unit 100. The controlunit 100 supplies control signals (clock, lateral/longitudinalsynchronization signal, and others) to all the other units of the imagecapturing apparatus. That is, the control unit 100 controls operationsof all the other units in pipeline. The control unit 100 includes anoise-level detecting unit 102 that detects a noise level of an imagebased on a gain of an AGC circuit 120 and a luminance signal obtained atan MTF correcting unit 160 and a switching-signal generating unit 104that generates a switching signal for enabling and disabling an IIRfilter and an FIR filter to provide their functions, which will bedescribed later, in the MTF correcting unit 160 based on a result ofdetection performed by the noise-level detecting unit 102.

The image capturing apparatus includes an imaging device 110. Theimaging device 110 includes elements, e.g., a charge-coupled device(CCD) or a complementary metal oxide semiconductor (CMOS) sensor andconverts an optical image captured by using a wide-angle optical system(not shown) into electric signals (pixel signals). The wide-angleoptical system has large magnification aberration and distortion. Theimaging device 110 further includes a Bayer-array color filter andsequentially outputs RGB pixel signals in a Bayer array according tocoordinate values (x, y) fed from the control unit 100. The control unit100 also sequentially feeds the coordinate values (x, y), which it feedsto the imaging device 110, to the units arranged at a subsequent stageof the control unit 100 with a predetermined time lag.

In an alternative configuration, instead of the control unit 100, theimaging device 110 generates the coordinate values (x, y) in response toreception of clock and lateral/longitudinal synchronization signals andsequentially feeds them to the units arranged at a subsequent stage ofthe control unit 100.

The image capturing apparatus includes the AGC circuit 120 at asubsequent stage of the imaging device 110. The AGC circuit 120amplifies a pixel signal, which is an analog signal, output from theimaging device 110 to a predetermined level. The gain of the AGC circuit120 is set to an appropriate value by making tradeoffs between arequired lightness of an image frame and a noise level. As will bementioned later, the noise-level detecting unit 102 in the control unit100 detects a noise level of an image based on the gain of this AGCcircuit 120.

The image capturing apparatus includes an analog-to-digital (A/D)converter 130 at a subsequent stage of the AGC circuit 120. TheBayer-array RGB image signals output from the AGC circuit 120 are analogsignals. The A/D converter 130 converts the analog Bayer-array RGB imagesignals into digital signals (pixel data). Each of the digital signalsis a signal with, for example, 8 bits per color of RGB.

The image capturing apparatus includes a Bayer interpolation unit 140 ata subsequent stage of the A/D converter 130. The Bayer interpolationunit 140 receives the digital Bayer-array RGB image signal (pixel data),generates pixel data for all coordinate positions by performing linearinterpolation on an individual color basis of RGB.

Although, in the present embodiment, the imaging device has beendiscussed as including the Bayer-array color filter, the presentembodiment is also effective for an imaging device that includes a colorfilter of another arrangement, such as another CMYG array or anRGB-and-infrared-ray (Ir) array. Particularly, an imaging device thatincludes such a color filter that has a four-color array as mentionedearlier requires, as compared to a type of three colors such as RGB, amemory of lower latency or a 4-port random access memory (RAM) forcorrection of magnification chromatic aberration.

The image capturing apparatus includes amagnification-chromatic-aberration correcting unit 150 at a subsequentstage of the Bayer interpolation unit 140. Themagnification-chromatic-aberration correcting unit 150 receives theBayer-interpolated R, G, and B pixel data, performs coordinatetransformation (coordinate transformation for magnification chromaticaberration) on an individual chrominance component basis of RGB by usinga predetermined polynomial, and outputs RGB pixel data having undergonethe magnification chromatic aberration correction. As will be mentionedlater, the coordinate transformation for correcting the magnificationchromatic aberration can be performed by using a memory that has arelatively small capacity and low latency or a memory (e.g., a staticrandom access memory (SRAM)) that has a relatively small capacity and aplurality of ports.

The image capturing apparatus includes an MTF correcting unit 160 at asubsequent stage of the magnification-chromatic-aberration correctingunit 150. The MTF correcting unit 160 includes, as will be mentionedlater, an FIR filter serving as an edge enhancement filter and an IIRfilter serving as a noise reduction filter. The MTF correcting unit 160receives an input of RGB pixel data having undergone the magnificationchromatic aberration correction, converts the RGB pixel data intoluminance signals Y and chrominance signals Cb and Cr, and thereafter,under a normal condition, performs high-frequency enhancement (edgeenhancement) of the Y signals by using the FIR filter; however, under acondition where the noise level of an image has been increased, performsnoise reduction of the YCbCr signals by using the IIR filter, convertsYCbCr signal having undergone either the edge enhancement or the noisereduction back into RGB signals (RGB pixel data), and outputs the RGBsignals. The present invention is directed to the FIR filter and the IIRfilter of this MTF correcting unit 160. According to an embodiment, aswill be mentioned later, the noise-level detecting unit 102 in thecontrol unit 100 obtains average lightness of an image based on theluminance signals Y obtained by the MTF correcting unit 160, therebydetecting a noise level.

The image capturing apparatus includes a distortion correcting unit 170at a subsequent stage of the MTF correcting unit 160. The distortioncorrecting unit 170 receives the RGB pixel data having undergone themagnification chromatic aberration correction and the MTF correction,performs coordinate transformation (coordinate transformation fordistortion) of the RGB chrominance components collectively by using apredetermined polynomial or the like, and outputs RGB pixel data havingundergone the distortion correction. As will be mentioned later, amemory for use in the coordinate transformation for the distortioncorrection desirably has a larger capacity (by one image frame atmaximum) than a memory for use in the magnification chromatic aberrationcorrection; however, the number of ports required of the memory forcorrecting the distortion is one. Therefore, as this memory, a memory(dynamic random access memory (DRAM) or the like) of high latency can beused.

The image capturing apparatus includes a gamma correcting unit 180 at asubsequent stage of the distortion correcting unit 170. The gammacorrecting unit 180 receives the RGB pixel data output from thedistortion correcting unit 170, performs predetermined gamma correctionon the data by using lookup tables or the like on an individual colorbasis of RGB, and outputs RGB pixel data having undergone the gammacorrection. The pixel data output from the gamma correcting unit 180 ismonitor-displayed on a display unit (not shown).

The image capturing apparatus having the configuration illustrated inFIG. 1 can provide an imaging system of high-image-quality that is smallin circuit scale and of low cost can be provided even when the systemuses a high-view-angle optical system that causes chromaticmagnification aberration and distortion to occur. Even when a noiselevel of an image increases due to an increase in gain in a darkportion, the noise level can be suppressed. Meanwhile, themagnification-chromatic-aberration correcting unit 150 can be replacedwith a magnification-chromatic-aberration-and-distortion correcting unitthat corrects magnification chromatic aberration and distortionsimultaneously. When such amagnification-chromatic-aberration-and-distortion correcting unit isemployed, the distortion correcting unit 170 becomes unnecessary. Thegamma correcting unit 180 can be arranged at an immediately subsequentstage of the Bayer interpolation unit 140.

Specific exemplary configurations of themagnification-chromatic-aberration correcting unit 150, the MTFcorrecting unit 160, and the distortion correcting unit 170 will bedescribed in detail below.

Prior to giving a detailed description about themagnification-chromatic-aberration correcting unit 150 and thedistortion correcting unit 170, principles of magnification chromaticaberration correction and distortion correction will be described.

As schematically illustrated in FIG. 2, when an image is captured byusing an optical system and when magnification chromatic aberration anddistortion occurs, pixel data pertaining to an original position (pixel)at the upper right of an image frame indicated by 1 is shifted from thisoriginal position due to distortion, and further shifted differentlyamong different chrominance components of RGB due to magnificationchromatic aberration. Accordingly, an R component, a G component, and aB component actually imaged by the imaging device are shown in FIG. 2 as2(R), 3(G), and 4(B), respectively. Magnification chromatic aberrationcorrection and distortion correction can be performed by copying thepixel data pertaining to the RGB chrominance components at the positions(pixels) of 2(R), 3(G), 4(B) to the position (pixel) of 1, which is theoriginal position; i.e., by performing coordinate transformation. Inperforming the coordinate transformation, the positions 2, 3, and 4 areemployed as coordinate-transformation source coordinates while theposition 1 is employed as coordinate-transformation target coordinates.

The amount of distortion and the amount of magnification chromaticaberration can be obtained from design data of the optical system; andit is therefore possible to calculate amounts of shifts of the RGBchrominance components relative to the original position.

FIG. 3 is a schematic for explaining a method for correctingmagnification chromatic aberration and distortion simultaneously.Specifically, the magnification chromatic aberration and distortion canbe corrected simultaneously by copying the pixel data pertaining to theRGB chrominance components at the positions (pixels) of 2(R), 3(G), and4(B) to the position (pixel) of 1, which is the original position; inother words, by performing coordinate transformation. However, thismethod is disadvantageous in that in this method it is necessary toprovide a memory that has relatively large capacity and any one of lowlatency and multiple ports for each of RGB. For example, in the exampleillustrated in FIG. 3, a fast, 6-line memory becomes necessary for eachof RGB to perform the coordinate transformation.

FIGS. 4A and 4B are schematics for explaining the methods for correctingmagnification chromatic aberration and distortion independently.Although magnification chromatic aberration occurs at shift amounts thatdiffer among different chrominance components, the shift amounts arerelatively small. In contrast, distortion occurs with a relatively largeshift amount but the shift amount is same for the different chrominancecomponents. With an attention focused on this, coordinate transformationof pixel data is performed on an individual chrominance component basisof RGB (in the example to be mentioned later, RB chrominance componentsare subjected to coordinate transformation and copied to the position ofG component) to correct magnification chromatic aberration; thereafter,the RGB pixel data having undergone the magnification chromaticaberration correction is subjected, as one set of data, to coordinatetransformation for distortion correction. This method allows toseparately use memories for use in the coordinate transformation formagnification chromatic aberration correction and for the coordinatetransformation for distortion correction. More concretely, it ispossible to use a fast (low-latency or with multiple ports),small-capacity memory that is for RGB for use in the coordinatetransformation for magnification chromatic aberration correction and usea slow (high-latency or with a single port), large-capacity memory to beshared among RGB and for use in the distortion correction. Use ofseparate memories leads to cost reduction. The system configuration ofFIG. 1 is given for illustration of this.

Referring to FIG. 4A, which is a schematic of magnification chromaticaberration correction, pixel data pertaining to RB chrominancecomponents at the positions (pixels) 2(R) and 4(B) is subjected tocoordinate transformation to be copied to 3(G), which is the position(pixel) of a G component. The magnification chromatic aberrationcorrection is achieved by performing this operation. Referring to FIG.4B, which is a schematic of distortion correction, pixel data havingundergone the magnification chromatic aberration correction andpertaining to RGB chrominance components at the positions (pixels) 3 issubjected, as one set of data, to coordinate transformation to be copiedto the position (pixel) 1, which is the original position. Thedistortion correction is achieved by performing this operation.

In the example illustrated in FIGS. 4A and 4B, a a-line, fast memorythat processes RGB individually can be satisfactorily used for themagnification chromatic aberration correction. On the other hand, a5-line memory for the distortion correction is additionally required;however, this memory can be a slow memory to be shared among RGB, whichleads to total cost reduction as compared to FIG. 3.

The distortion discussed here denotes distortion of a lens in aprojection scheme to be used. Examples of the projection scheme to beused include a projection scheme for obtaining an image as viewed fromabove a camera and a projection scheme for displaying a portion of animage in an enlarged manner.

FIG. 5 is a schematic configuration diagram of themagnification-chromatic-aberration correcting unit 150. Themagnification-chromatic-aberration correcting unit 150 includescoordinate-transformation memories (line buffers) (SRAM) for correctingmagnification chromatic aberration, in which 1510(R), 1510(G), and1510(B) are for an R chrominance component, a G chrominance component,and a B chrominance component, respectively; a coordinate transformationunit 1520 that calculates transformation coordinates for magnificationchromatic aberration correction on an individual color basis of RGBbased on a predetermined coordinate transformation algorithm; and acoordinate-transformation coefficient table 1530 that stores thereincoefficients for use in the coordinate transformation algorithm.

The magnification chromatic aberration correction can be satisfactorilyperformed with, as a line buffer, a memory having relatively smallcapacity and yet having either three ports for RGB or low latency. Inthis example, each of the coordinate transformation memories 1510(R),1510(G), and 1510(B) is assumed to include an SRAM with capacity of 20lines on an assumption that a maximum shift amount due to magnificationchromatic aberration is 20 lines. The size of the memory in theX-direction depends on the resolution. For example, the size in theX-direction of 640 dots is sufficient when the resolution is equivalentto that of video graphics array (VGA) (640×480). The color depth is 8bits per color of RGB, and writing and reading from and to each of thecoordinate transformation memories 1510(R), 1510(G), and 1510(B) isperformed in an 8-bit unit.

Thus, each of the coordinate transformation memories 1510(R), 1510(G),and 1510(B) is small in capacity; therefore, each of the memoriesdesirably includes a 3-port SRAM provided in an image processing chip toensure a memory area to contain the 20 lines. When the memory is alow-latency memory such as SRAM, a 1-port memory can be used as a 3-portmemory in a time sharing manner.

The pixel data of individual colors of RGB having undergone themagnification chromatic aberration is sequentially written to acorresponding one of the coordinate transformation memories 1510(R),1510(G), and 1510(B) from its first line according to correspondingcoordinate values (x, y). When 20 lines of pixel data has been writtento each of the memories, pixel data is discarded sequentially from thefirst line and subsequent lines of pixel data are sequentially newlywritten to take place of the discarded data. Thus, RGB pixel data, of 20lines at maximum per memory, that is necessary to perform the coordinatetransformation for correcting the magnification chromatic aberration issequentially stored in each of the coordinate transformation memories1510(R), 1510(G), and 1510(B).

The coordinate values (x, y) indicate a read-out position of one frameof a captured image. Meanwhile, each of the coordinate transformationmemories 1510(R), 1510(G), and 1510(B) is 20-line line buffer in which aline to be written cyclically changes; therefore, it is useless to usethe coordinate values (x, y) directly as write addresses on thecoordinate transformation memories 1510(R), 1510(G), and 1510(B).Therefore, it is necessary to translate the coordinate values (x, y)into real addresses on the coordinate transformation memories 1510(R),1510(G), and 1510(B); however, the configuration for this is notillustrated in FIG. 5. The same goes for the relation in readingoperation between post-transformation coordinate values (X, Y) and readaddresses on the coordinate transformation memories 1510(R), 1510(G),and 1510(B), which will be described later.

The coordinate transformation unit 1520 receives the coordinate values(x, y), which are coordinate-transformation target coordinates,calculates transformation coordinates for magnification chromaticaberration correction on an individual color basis of RGB by usingpredetermined coordinate transformation algorithm, such as polynomial,and outputs coordinate values (X, Y), which arecoordinate-transformation source coordinates on an individual colorbasis of RGB. As illustrated in FIG. 4A, in the present embodiment, Rand B chrominance components are subjected to coordinate transformationto be copied to the position of a G component. Accordingly, while, withregard to the G component, the coordinate transformation unit 1520 thatreceives an input the coordinate values (x, y) outputs them as thecoordinate values (X, Y) without performing transformation, with regardto the R and B chrominance components, the coordinate transformationunit 1520 transforms the thus-input coordinate values (x, y) on anindividual color basis of RB by using predetermined coordinatetransformation algorithm into the coordinate values (X, Y) and outputsthe coordinate values (X, Y). This operation is repeated for each set ofcoordinate values (x, y).

When an origin is assumed at the center of an image frame, thecoordinate transformation algorithm can be expressed with followingEquation (1):X=x+[a(1)+a(2)×abs(x)+a(3)×abs(y)+a(4)×y ² ]×xY=y+[b(1)+b(2)×abs(y)+b(3)×abs(x)+b(4)×x ² ]×y  (1)

where abs( ) is an absolute value of the parameter in ( ), and a(1) toa(4) and b(1) to b(4) are coordinate transformation coefficients. Thecoordinate transformation coefficients are stored in thecoordinate-transformation coefficient table 1530 in advance.

In parallel (in actual, with a delay of a predetermined period of time)with the writing to the coordinate transformation memories 1510(R),1510(G), 1510(B) mentioned earlier, the RGB pixel data is sequentiallyread from the coordinate transformation memories 1510(R), 1510(G),1510(B) according to the coordinate values (X, Y) output from thecoordinate transformation unit 1520 (in actual, address-translatedvalues of the coordinate values (X, Y)). In this case, G-component pixeldata is read from the coordinate transformation memory 1510(G) at thesame position as that where the G-component pixel data has been written.In contrast, each of R-component pixel data and B-component pixel datais read from a corresponding one of the coordinate transformationmemories 1510(R) and 1510(B) at a position shifted from the positionwhere the chrominance component pixel data has been written by apredetermined distance; i.e., by an amount of magnification chromaticaberration.

By performing the above operations, the RGB pixel data having undergonethe magnification chromatic aberration correction is output from thecoordinate transformation memories 1510(R), 1510(G), and 1510(B).Specifically, the RGB pixel data at the coordinate-transformation sourcecoordinate values (X, Y) are output as the RGB pixel data at thecoordinate-transformation target coordinate values (x, y).

FIGS. 6A to 6C illustrate various exemplary configurations of thecoordinate transformation unit 1520. FIG. 6A is an exemplaryconfiguration where a chrominance component for G is not subjected tocoordinate transformation and coordinate values (x, y), which are inputvalues, are output as coordinate values (X, Y) for G, while chrominancecomponents for R and B are subjected to coordinate transformationperformed by a coordinate-transformation computation unit 1521 for R anda coordinate-transformation computation unit 1522 for (B), respectively,that transform the coordinate values (x, y), which are the input values,to output coordinate values (X, Y) for R and coordinate values (X, Y)for B. Because the coordinate-transformation computation units areprovided only for the R and B chrominance components, it is possible tosuppress the circuit scale.

FIGS. 6B and 6C illustrate other exemplary configurations devised with afocus given to a fact that R and B chrominance components are generallyshifted by magnification chromatic aberration to be substantiallysymmetrical about a G chrominance component (FIG. 2). FIG. 6Billustrates an exemplary configuration where onecoordinate-transformation computation unit 1523 calculates correctionamounts for the coordinate values (x, y), a subtracting unit 1524subtracts the correction amounts from the coordinate values (x, y) toobtain coordinate values (X, Y) for B, while an adding unit 1525 addsthe correction amounts to the coordinate values (x, y) to obtaincoordinate values (X, Y) for R. On the other hand, the input coordinatevalues (x, y) for G are output as they are as the coordinate values (X,Y) for G in the same manner as that shown in FIG. 6A.

FIG. 6C depicts an exemplary configuration where a gain circuit 1526 isprovided to adjust the correction amounts for R to allow for deviationbetween symmetrical positions. The exemplary configurations illustratedin FIGS. 6B and 6C have been embodied with only onecoordinate-transformation computation unit, leading to further reductionin circuit scale.

A look-up table (LUT) that stores therein correspondence between inputcoordinate values (x, y) and output coordinate values (X, Y) for each ofR and B chrominance components can be provided in place of thecoordinate-transformation computation units 1521 and 1522 illustrated inFIG. 6A so that coordinate-transformation source coordinate values (X,Y) corresponding to coordinate-transformation target coordinate values(x, y) can be directly obtained by using the LUT. Similarly, an LUT thatstores therein correspondence between input coordinate values (x, y) andcorrection amounts can be provided in place of thecoordinate-transformation computation unit 1523 illustrated in FIGS. 6Band 6C so that correction amounts corresponding to coordinate values (x,y) can be directly obtained by using the LUT. This allows omittingcalculations for coordinate transformation, thereby making magnificationchromatic aberration correction implementable basically only on memorychip.

FIG. 7 is a detailed configuration diagram of the distortion correctingunit 170. The distortion correcting unit 170 includes an RGB combiningunit 1710 that combines three pieces, each corresponding to one color,of RGB pixel data into one data piece; a coordinate transformationmemory 1720 (SRAM) that is to be shared among the chrominance componentsof the RGB pixel data and for use in the distortion correction; an RGBseparating unit 1730 that separates the combined RGB pixel data intooriginal chrominance components; acoordinate-transformation-for-correcting-distortion computation unit1740 that calculates transformation coordinates for the distortioncorrection of the combined RGB pixel data by using predeterminedcoordinate transformation algorithm; and a coordinate-transformationcoefficient table 1750 that stores therein coefficients to be used withthe coordinate transformation algorithm.

Because distortion occurs with a relatively large shift amount, a buffermemory for storing therein pixel data of one image frame at maximum isdesirably used to perform the distortion correction. Meanwhile, becausethe RGB chrominance components are shifted by a single shift amount, asingle buffer memory having a bit width equal to a total bit number ofthe RGB pixel data can be satisfactorily employed. It is assumed thatresolution is of the VGA (640×480), the number of bits (color depth) ofthe RGB pixel data is 8 bits per color of RGB, and the coordinatetransformation memory 1720 is a DRAM to and from which writing andreading is performed in a unit of 24-bit, 640×480 dots.

The coordinate transformation memory 1720 that requires a considerablylarge capacity as mentioned above is difficult to be embodied in theform of an SRAM in an image processing chip in view of cost, and a1-port memory can be satisfactorily used to handle RGB; therefore, thecoordinate transformation memory 1720 is desirably embodied by using aDRAM provided outside the image processing chip.

The RGB combining unit 1710 receives the RGB pixel data (8 bits each)having undergone the magnification chromatic aberration correction,sequentially combines the RGB pixel data into one piece of pixel data(24 bits), and outputs the pixel data. The thus-combined RGB pixel datais sequentially written to the coordinate transformation memory 1720from its first line according to coordinate-transformation targetcoordinate values (x, y).

Meanwhile, the coordinate-transformation-for-correcting-distortioncomputation unit 1740 receives the coordinate-transformation targetcoordinate values (x, y), calculates transformation coordinates, whichare common to RGB, for the distortion correction by using apredetermined coordinate transformation algorithm, such as polynomial,and outputs coordinate-transformation source coordinate values (X, Y).The coordinate transformation algorithm can be expressed as Equation(1), which is the same as that for use in the magnification chromaticaberration correction mentioned earlier. As a matter of course,different coordinate transformation coefficients are to be used. Thecoordinate transformation coefficients are stored in thecoordinate-transformation coefficient table 1750 in advance.

In parallel (to be precise, with a delay of a predetermined period oftime) with the writing of the combined RGB pixel data (24 bits) to thecoordinate transformation memory 1720 mentioned earlier, combined RGBpixel data is sequentially read from the coordinate transformationmemory 1720 according to the coordinate values (X, Y) output from thecoordinate-transformation-for-correcting-distortion computation unit1740. The RGB separating unit 1730 separates the combined RGB pixel data(24 bits) read from the coordinate transformation memory 1720 into itsoriginal pixel data of individual R, G, and B components (8 bits each).

As a result of these operations, R pixel data, G pixel data, and B pixeldata having undergone the distortion correction are output from the RGBseparating unit 1730. Put another way, the R pixel data, the G pixeldata, and the B pixel data are copied to the coordinate values (x, y),or their original position.

Also in the case of the distortion correction, an LUT that storestherein correspondence between input coordinate values (x, y) and outputcoordinate values (X, Y) can be provided so thatcoordinate-transformation source coordinate values (X, Y) correspondingto coordinate-transformation target coordinate values (x, y) aredirectly obtained by using the LUT. This allows omitting calculationsfor coordinate transformation, thereby making distortion correctionimplementable basically only on memory chip as well.

The MTF correcting unit 160 will be described below. As illustrated inFIG. 8, the MTF correcting unit 160 includes an RGB/YCbCr conversionunit 1610, a filtering unit 1620, and a YCbCr/RGB conversion unit 1630.

The RGB/YCbCr conversion unit 1610 receives an input of the RGB pixeldata and separates the RGB pixel data into luminance signals Y andchrominance signals Cb and Cr by using, for example, the followingequations:Y=0.299R+0.587G+0.114B  (2)Cr=0.500R−0.419G−0.081B  (3)Cb=−0.169R−0.332G+0.500B  (4)

The filtering unit 1620 includes an edge enhancement filter (FIR filter)and a noise reduction filter (IIR filter), and performs, under a normalcondition, high-frequency enhancement (edge enhancement) of theluminance signals Y by using the FIR filter; however, when the noiselevel in an image has been increased, performs noise reduction of theYCbCr signals by using the IIR filter. A feature of the presentinvention resides in the configuration of the filtering unit 1620. Aspecific configuration and operations of the filtering unit 1620 will bedescribed in detail later.

The YCbCr/RGB conversion unit 1630 receives an input of the YCbCrsignals having undergone any one of the high-frequency enhancement andthe noise reduction, converts the signals back into RGB pixel data byusing the following equations for example, and outputs the RGB pixeldata:R=Y+1.402Cr  (5)G=Y−0.714Cr−0.344Cb  (6)B=Y+1.772Cb  (7)

The YCbCr/RGB conversion unit 1630 can be omitted when luminance signalsY and chrominance signals YCbCr, rather than RGB signals, are desirablyoutput to a subsequent stage.

FIG. 9 is a schematic diagram of the specific configuration of thefiltering unit 1620 in the MTF correcting unit 160. As illustrated inFIG. 9, the filtering unit 1620 includes an IIR filter 1621 serving as anoise reduction filter, an FIR filter 1622 serving as an edgeenhancement filter, a line buffer 1623 to be used by both the IIR filter1621 and the FIR filter 1622, and switches SW1 and SW2.

Meanwhile, a switching signal is fed to each of the switches SW1 and SW2from the switching-signal generating unit 104 (FIG. 1) in the controlunit 100. How the switching signal is generated will be described later.Under a normal condition where a noise level of an image is relativelylow, the switch SW1 disables the IIR filter 1621 so that the YCbCrsignals that are input are directly sent to the line buffer 1623 whilethe switch SW2 enables the FIR filter 1622 so that the YCbCr signals aresent from the FIR filter 1622 out through the output terminal. Incontrast, when a noise level of an image is relatively high, the switchSW1 enables the IIR filter 1621 so that YCbCr signals are sent from theIIR filter 1621 to the line buffer 1623 while the switch SW2 disablesthe FIR filter 1622 so that the YCbCr signals are directly sent from theline buffer 1623 out through the output terminal.

The line buffer 1623 is used by both the IIR filter 1621 for use innoise reduction and the FIR filter 1622 for use in edge enhancement. Forthe FIR filter 1622 of which number of the taps is set to, for example,5×5, a line buffer with capacity of 5 lines or larger can besatisfactorily used as the line buffer 1623. Meanwhile, the line buffer1623 stores therein 1 pixel of YCbCr signals (YCbCr data) at eachaddress, to which, for example, 24 bits (1 word) are assigned to eachaddress, where a 24-bit signal is formed with RGB components whose colordepth is 8 bits each; i.e., YCbCr components of 8 bits each.

Under a normal condition where a gain of the AGC circuit 120 isrelatively low, i.e., where an image has a relatively low noise, the FIRfilter 1622 enhances high-frequency components of the image (edgeenhancement) that are attenuated by an optical system, therebyperforming shaping of spatial frequency characteristics. FIG. 10illustrates example coefficients set to the FIR filter 1622. The FIRfilter 1622 sequentially reads YCbCr signals of 5×5 pixels, at thecenter of which a target pixel is located, on the (N−2)th to (N+2)thlines from the line buffer 1623 and performs edge enhancement filteringon the Y signal of the target pixel. This prevents increase in the noiselevel of the chrominance signals CbCr.

FIG. 11 is a detailed configuration diagram of the FIR filter 1622. Aseparating unit 16221 reads YCbCr signals from the line buffer 1623 asan input and separates the YCbCr signals into Y signals and CbCrsignals. A filter 16222 performs edge enhancement of the Y signals byusing such coefficients as given in FIG. 10. A combining unit 16223combines the signals having undergone the edge enhancement and the CbCrsignals together to output YCbCr signals.

Returning to FIG. 9, when the noise level in an image has been increaseddue to an increase in gain of the AGC circuit 120 in a dark portion orthe like, the IIR filter 1621 performs noise reduction. Coefficients forthe IIR filter 1621 can be set, for example, using following Equations(8) to (10):Cb(x,y)=0.25*Cb(x,y)+0.375*Cb(x,y−1)+0.375*Cb(x−1,y)  (8)Cr(x,y)=0.25*Cr(x,y)+0.375*Cr(x,y−1)+0.375*Cr(x−1,y)  (9)Y(x,y)=0.5*Y(x,y)+0.25*Y(x,y−1)+0.25*Y(x−1,y)  (10)

Cb(x, y) and Cr(x, y) are chrominance signals of a target pixel atcoordinates (x, y) on the Nth line; and Y(x, y) is a luminance signal atthe same coordinates (x, y). Cb(x, y−1) and Cr(x, y−1) are chrominancesignals at coordinates (x, y−1) on the (N−1)th line, which is the lineimmediately preceding the Nth line where the target pixel is located;and Y(x, y−1) is a luminance signal at the same coordinates (x, y−1).Cb(x−1, y) and Cr(x−1, y) are chrominance signals on the Nth line atcoordinates (x−1, y), which are coordinates immediately preceding thecoordinates of the target pixel; and Y(x−1, y) is a luminance signal atthe same coordinates (x−1, y).

The IIR filter 1621 sequentially reads YCbCr signals at coordinates (x,y−1) on the (N−1)th line and (x−1, y) on the Nth line from the linebuffer 1623 and performs noise reduction filtering on the these signalsand the YCbCr signals at coordinates (x, y) of the target pixel that areinput. The thus-processed YCbCr signals are written to the line buffer1623 and fed back to the IIR filter 1621 at future occasions when thetarget pixel is processed.

FIG. 12 is a detailed configuration diagram of the IIR filter 1621. Aseparating unit 16211 separates each of the YCbCr signals that are inputand the YCbCr signals that are fed back from the line buffer 1623 intoY, Cb, and Cr. A filter 16213, a filter 16214, and a filter 16212perform filtering of the Cb signals, the Cr signals, and the Y signalsby applying Equation (8), Equation (9), and Equation (10) thereto,respectively. Equations (8), (9), and (10) exert relatively strong noisesuppression on the CbCr signals while the same exert relatively weaknoise suppression on the Y signals. Note that Equations (8), (9), and(10) are given for illustration, and a configuration that performs nofiltering on the Y signals can alternatively be employed. A combiningunit 16215 combines the Cb, Cr, and Y signals having undergone the noisereduction together.

Referring to FIG. 13 and FIG. 14, an overview of process procedure ofthe filtering unit 1620 will be described below.

FIG. 13 illustrates how the IIR filter 1621, the line buffer 1623, andthe FIR filter 1622 are connected together for a case where a gain ofthe AGC circuit 120 is relatively low; i.e., a noise level of an imageis relatively low. In this case, the IIR filter 1621 is disabled(output: open) and the FIR filter 1622 is enabled. The switch SW1 causesthe YCbCr signals output from the RGB/YCbCr conversion unit 1610 tobypass the IIR filter 1621 to be sequentially written to the line buffer1623. The FIR filter 1622 sequentially reads YCbCr signals of 5×5 pixels(pixels at coordinates (x−2, y−2) to (x+2, y+2)) on the (N−2)th to(N+2)th lines from the line buffer 1623 and uses Y signals of thesesignals to perform edge enhancement filtering of the luminance signal Yof the target pixel at coordinates (x, y) by applying coefficients givenin FIG. 10 thereto. The YCbCr signals, of which Y signals havingundergone the edge enhancement, are sequentially sent to the YCbCr/RGBconversion unit 1630 by way of the switch SW2 to be converted back intoRGB signals.

FIG. 14 illustrates how the IIR filter 1621, the line buffer 1623, andthe FIR filter 1622 are connected together in a case where the noiselevel in an image has increased due to an increase in gain of the AGCcircuit 120. In this case, the IIR filter 1621 is enabled and the FIRfilter 1622 is disabled (output: open).

The YCbCr signals (YCbCr signals at coordinates (x, y)) output from theRGB/YCbCr conversion unit 1610 are sequentially input to the IIR filter1621. The processed YCbCr signals at coordinates (x, y−1) on the (N−1)thline and the processed YCbCr signals at coordinates (x−1, y) on the Nthline are sequentially read from the line buffer 1623 and fed back to theIIR filter 1621. The IIR filter 1621 receives an input of the YCbCrsignals at coordinates (x, y) and the processed YCbCr signals atcoordinates (x, y−1) and (x−1, y) fed back from the line buffer 1623,and performs noise reduction filtering of the YCbCr signals atcoordinates (x, y) of the target pixel by using Equations (8), (9), and(10). The YCbCr signals having undergone the noise reduction aresequentially written to the line buffer 1623 by way of the switch SW1.Simultaneously, the line buffer 1623 sequentially reads the processedYCbCr signals at coordinates (x, y) and the processed YCbCr signals atcoordinates (x, y−1) and coordinates (x−1, y). The YCbCr signals atcoordinates (x, y) read out from the line buffer 1623 are sequentiallysent to the YCbCr/RGB conversion unit 1630 by way of the switch SW2while bypassing the FIR filter 1622 to be converted back into RGBsignals. The YCbCr signals at coordinates (x, y−1) and (x−1, y) read outfrom the line buffer 1623 are fed back to the IIR filter 1621 to be usedin noise reduction of a subsequent target pixel.

As mentioned earlier, depending on switching signals fed from theswitching-signal generating unit 104 of the control unit 100, thefiltering unit 1620 takes either the configuration illustrated in FIG.13 (the IIR filter 1621 is disabled and the FIR filter 1622 is enabled),which is for a normal condition, to perform edge enhancement onluminance signals placing priority on resolution or the configurationillustrated in FIG. 14 (the IIR filter 1621 is enabled and the FIRfilter 1622 is disabled), which is for a case where the noise level inan image has increased due to an increase in gain of the AGC circuit 120in a dark portion or the like, to perform noise reduction of YCbCrsignals. Meanwhile, the IIR filter 1621 performs feedback of theprocessed YCbCr signals by using a line buffer. Using the line buffer1623, which is used by the FIR filter 1622, also as the line buffer foruse in the feedback in a sharing manner eliminates the need of providingan additional line buffer for the IIR filter, leading to reduction incircuit scale.

Detection of a noise level of an image and generation of a switchingsignal will be described below. The noise-level detecting unit 102 ofthe control unit 100 employs, for example, one of the following methodsof detecting a noise level of an image. As a matter of course, these aredescribed for illustration only, and any method of detecting a noiselevel can be employed.

(i) Output signals of the imaging device 110 are amplified in the AGCcircuit 120 prior to be subjected to A/D conversion performed by the A/Dconverter 130. A gain of the AGC circuit 120 is set to an appropriatevalue by making tradeoffs between a required lightness of an image frameand a noise level; however, when a decrease in lightness in a darkportion causes the gain to be a certain value or higher, the noise levelundesirably increases to be equal to or above an allowable range.Specifically, when image capturing is performed with gain increased toincrease sensitivity to a dark portion, noise becomes dominant in acaptured image. By utilization of this fact, the noise-level detectingunit 102 can determine a noise level based on a gain of the AGC circuit120.

(ii) Generally, the AGC circuit 120 controls its gain so as to maintainlightness of an image captured by the imaging device 110 constant.Therefore, average lightness is constant under a normal condition;however, if sensitivity is insufficient even when the gain is increasedto its maximum value, average lightness of an image decreases.Therefore, whether a gain has reached its maximum value, i.e., whether anoise level has increased, can be determined by detecting averageluminance of an image frame. By utilization of this fact, thenoise-level detecting unit 102 can calculate, based on luminance signalsY obtained by the MTF correcting unit 160, average luminance of an imageframe from a sum of luminance values taken across the entire image frameor, in some case, a sum of weighted luminance values, in which weightsare assigned to signals corresponding to a subject at an image-framecenter or the like, thereby determining a noise level.

(iii) Brightness is generally inversely proportional to a noise level.By utilization of this fact, an illuminance sensor can be provided sothat the noise-level detecting unit 102 can determine a noise levelbased on an output of the illuminance sensor.

The switching-signal generating unit 104 generates switching signalsbased on a result of detection of the noise-level detecting unit 102.Specifically, when a noise level of an image is relatively low,switching signals that cause the switch SW1 to disable the IIR filter1621 and the switch SW2 to enable the FIR filter 1622 are generated; incontrast, when the noise level is relatively high, switching signalsthat cause the switch SW1 to enable the IIR filter 1621 and the switchSW2 to disable the FIR filter 1622 are generated.

The embodiment of the present invention has been described. As a matterof course, the present invention can be implemented by configuringprocessing functions of the image processing apparatus illustrated inFIG. 1, or the like, as computer programs and causing a computer toexecute the computer programs. Alternatively, the present invention canbe implemented by configuring process procedure thereof as computerprograms and causing a computer to execute the computer programs. Thecomputer programs for causing a computer to carry out the processingfunctions can be stored and/or provided by recording the computerprograms in a computer-readable recording medium, such as a flexibledisk (FD), a magneto-optical disk (MO), a read only memory (ROM), amemory card, a compact disc (CD), a digital versatile disc (DVD), and aremovable disk, and/or distributed via a network such as the Internet.

The invention claimed is:
 1. An image processing apparatus thatprocesses image data obtained by an imaging device, the image processingapparatus comprising: a line buffer that temporarily and sequentiallystores therein the image data; a finite impulse response (FIR) filterthat performs shaping of spatial frequency characteristics of the imagedata by using the line buffer; an infinite impulse response (IIR) filterthat uses the same line buffer that is used by the FIR filter as a linebuffer for use in feedback of processed image data; a noise-leveldetecting device that detects a noise level of the image data; aswitching-signal generating device that generates a switching signalbased on the noise level detected at the noise-level detecting device;at least one switching device configured to enable and disable the FIRfilter and the IIR filter independently according to the switchingsignal generated at the switching-signal generating device, and whereinwhen a noise level is low, the switching device switches the IIR filterOFF so that an input signal is directly sent to the line buffer, whilethe switching device switches the FIR filter ON so that a signal sentfrom the FIR filter is sent to an output terminal, and when a noiselevel is high, the switching device switches the IIR filter ON so that asignal sent from the IIR filter is sent to the line buffer, while theswitching device switches the FIR filter OFF so that a signal sent fromthe line buffer is directly sent to the output terminal.
 2. The imageprocessing apparatus according to claim 1, wherein the FIR filter is anedge enhancement filter that enhances high-frequency components of theimage data.
 3. The image processing apparatus according to claim 1,wherein the IIR filter is a low-pass filter that reduces noise.
 4. Theimage processing apparatus according to claim 1, wherein the noise-leveldetecting device detects the noise level based on a gain of an automaticgain control (AGC) circuit of the image processing apparatus.
 5. Theimage processing apparatus according to claim 1, wherein the noise-leveldetecting device detects the noise level based on average lightness ofan image.
 6. The image processing apparatus according to claim 1,wherein when the noise level is relatively low, the FIR filter isenabled and the IIR filter is disabled with an output terminal of theline buffer connected to an input terminal of the FIR filter, and whenthe noise level is relatively high, the FIR filter is disabled and theIIR filter is enabled with an output terminal of the IIR filterconnected to an input terminal of the line buffer and the outputterminal of the line buffer connected to an input terminal of the IIRfilter.
 7. An on-vehicle camera apparatus comprising: a wide-view-angleoptical system; an imaging device that reads an image captured throughthe optical system; the image processing apparatus according to claim 1,wherein the image processing apparatus processes the image data capturedby the imaging device; and a display device that displays image dataprocessed by the image processing apparatus.
 8. An on-vehicle cameraapparatus comprising: a wide-view-angle optical system; an imagingdevice that reads an image captured through the optical system; theimage processing apparatus according to claim 2, wherein the imageprocessing apparatus processes the image data captured by the imagingdevice; and a display device that displays image data processed by theimage processing apparatus.
 9. An on-vehicle camera apparatuscomprising: a wide-view-angle optical system; an imaging device thatreads an image captured through the optical system; the image processingapparatus according to claim 3, wherein the image processing apparatusprocesses the image data captured by the imaging device; and a displaydevice that displays image data processed by the image processingapparatus.
 10. An on-vehicle camera apparatus comprising: awide-view-angle optical system; an imaging device that reads an imagecaptured through the optical system; the image processing apparatusaccording to claim 1, wherein the image processing apparatus processesthe image data captured by the imaging device; and a display device thatdisplays image data processed by the image processing apparatus.
 11. Anon-vehicle camera apparatus comprising: a wide-view-angle opticalsystem; an imaging device that reads an image captured through theoptical system; the image processing apparatus according to claim 4,wherein the image processing apparatus processes the image data capturedby the imaging device; and a display device that displays image dataprocessed by the image processing apparatus.
 12. An on-vehicle cameraapparatus comprising: a wide-view-angle optical system; an imagingdevice that reads an image captured through the optical system; theimage processing apparatus according to claim 5, wherein the imageprocessing apparatus processes the image data captured by the imagingdevice; and a display device that displays image data processed by theimage processing apparatus.
 13. An on-vehicle camera apparatuscomprising: a wide-view-angle optical system; an imaging device thatreads an image captured through the optical system; the image processingapparatus according to claim 6, wherein the image processing apparatusprocesses the image data captured by the imaging device; and a displaydevice that displays image data processed by the image processingapparatus.